Provisioning high-availability datacenter networks for full bandwidth communication

Keywords: Datacenter networks Valiant load balancing Full bandwidth communication Service availability Link failures Capacity allocation a b s t r a c t One critical challenge in datacenter network design is full bandwidth communication. Recent advances have enabled this communication paradigm based on the notion of Valiant load balancing (VLB). In this paper, we target full bandwidth communication among all servers, for all valid traffic patterns, and under k arbitrary link failures. We focus on two typical datacenter topologies, VL2 and fat-tree, and propose a mechanism to perform VLB on fat-tree. We develop the minimum link capacity required on both topologies, where edge and core links are handled separately. These results can help datacenter providers to provision their networks with guaranteed availability. Based on the results, we evaluate the minimum total link capacity required on each topology and characterize the capacity increase trend with k and with the total number of supported servers. These studies are important for datacenter providers to project their capital expenditures on datacenter design, upgrade, and expansion. Next, we compare the total link capacity between the two topologies. We find that given the same server scale, fat-tree requires less total capacity than does VL2 for small k. For large k, there exists a turning point at which VL2 becomes more capacity-efficient. Existing and emerging Internet applications, such as web search, video streaming, and social networking, are migrating towards the cloud computing paradigm, where user applications are run over a common datacenter infrastructure that consists of tens to hundreds of thousands of servers. In this new context, each user application job (e.g., MapReduce) is partitioned and assigned to various servers, far beyond the number a single server rack can hold. To enable local computation, extensive data exchanges are performed among servers that reside separately within a datacenter, contributing to a huge amount of communication traffic. A recent survey conducted by IDC [1] regarding the challenges of cloud services has shown that performance and availability are among the top concerns expressed by potential cloud users. Accordingly , service availability is listed as a leading obstacle to the growth of cloud computing [2]. In this paper, we consider how to provision a high-availability communication system within a datacenter, which is an important factor in determining service availability. Communication among servers is supported by a datacenter network, which typically consists of multiple tiers of switches and/or routers. Conventional datacenter networks …